Focus adjustment device and focus adjustment method

ABSTRACT

A focus adjustment device, that provides a plurality of detection areas in an imaging region, repeatedly generates ranging values for a physical object in the detection areas, and performs focus adjustment for the physical object based on the ranging values, including a processor that has a statistical processing section, a movement state estimating section, a prediction section, and a control section, wherein the statistical processing section subjects ranging values of the plurality of detection areas to statistical processing, the movement state estimating section calculates representative values that contain statistical dispersion based on the statistical processing, and estimates movement state of the physical object based on time-series change in the representative value, the prediction section predicts focal position regarding the physical object based on the movement state that has been estimated, and the control section performs focus adjustment based on the focal position that has been predicted.

CROSS-REFERENCE TO RELATED APPLICATIONS

Benefit is claimed, under 35 U.S.C. § 119, to the filing date of priorJapanese Patent Application No. 2021-079868 filed on May 10, 2021. Thisapplication is expressly incorporated herein by reference. The scope ofthe present invention is not limited to any requirements of the specificembodiments described in the application.

BACKGROUND OF THE INVENTION 1. Field of the Invention

The present invention relates to a focus adjustment device and a focusadjustment method that have a plurality of areas for ranging, repeatedlyobtain ranging values for a physical object, and perform focusadjustment based on the ranging values.

2. Description of the Related Art

With an automatic focus adjustment device such as a camera, in order tofocus on a subject that is moving detection of defocus amount goes oncontinuously, and moving body estimation computation is performed usinghistory information of that defocus amount, to predict position of asubject in the future (at the time of actual shooting). However, thereare cases where it is not possible to predict future position of asubject with good accuracy, due to various factors (for example, handshake of the photographer, framing error, crossing subject, predictioncomputation error, etc.). It should be noted that with actual shooting,when the photographer presses the release button fully to instructshooting, shooting is actually performed after time delays such asshutter lag etc.

In order to address the above described drawbacks, a focus adjustmentdevice that continuously detects defocus amount, performs moving bodyestimation computation using history information of the defocus amount,and predicts future position of a subject, is disclosed in, for example,Japanese patent laid-open No. 2009-210815 (hereafter referred to as“patent publication 1”). With this focus adjustment device, in a casewhere it has been determined that defocus amount of ranging areas thathave been selected are discontinuous in time, ranging areas in whichdefocus amounts are continuous are searched for based on defocus amountsof a plurality of ranging areas other than the ranging areas that havebeen selected. Then, a ranging area at the closest range is selectedagain from ranging areas for which defocus amount is continuous, andmoving body estimation computation is performed using the defocus amountof the ranging area that has been selected again.

Also, a focus adjustment device that divides a plurality of areas into aplurality of groups in accordance with a specified range based onvariation in focus state of the plurality of areas, selects anappropriate group from the plurality of groups depending on a mainsubject, and performs focus adjustment based on focus state belonging tothe group that has been selected, is disclosed on Japanese patentlaid-open No. 2009-175310 (hereafter referred to as “patent publication2”). However, there is no description in this patent publication 2 of AF(autofocus) that performs moving body prediction computation.

Defocus amount that has been detected using a phase difference AF methodincludes a certain degree of ranging error caused by degradation ofdegree of similarity of focus detection signals for left and right (orupper and lower) pupils due to low light or low contrast conditionsetc., and slight offset of in-focus position due to opticalcharacteristics, etc. As a ranging area selection method, generally, adefocus amount detecting the closest range is selected. However, in acase where ranging error is included in the defocus amount for theclosest range, or in a case where an incorrect ranging area is selectedif moving body estimation computation is performed using historyinformation, as disclosed in patent publication 1, accuracy of themoving body estimation computation is lowered. Also, with the methoddisclosed in patent publication 2, if a subject is moving it is easy foran incorrect ranging area to be selected, and for focus adjustmentprecision to be lowered.

SUMMARY OF THE INVENTION

The present invention provides a focus adjustment device and focusadjustment method that can improve precision of moving body estimationcomputation for a physical object that is moving.

A focus adjustment device of a first aspect of the present inventionprovides a plurality of detection areas in an imaging region formed byan optical system, repeatedly generates ranging values for a physicalobject in the detection areas, and performs focus adjustment for thephysical object based on the ranging values, and comprises a processorthat has a statistical processing section, a movement state estimatingsection, a prediction section, and a control section, wherein thestatistical processing section subjects ranging values of the pluralityof detection areas to statistical processing, the movement stateestimating section calculates representative values that containstatistical dispersion based on the statistical processing, andestimates movement state of the physical object based on time-serieschange in the representative values, the prediction section predictsfocal position for the physical object based on the movement state thathas been estimated, and the control section performs focus adjustmentbased on the focal position that has been predicted.

A focus adjustment method of a second aspect of the present inventionprovides a plurality of detection areas in an imaging region formed byan optical system, repeatedly generates ranging values for a physicalobject in the detection area, and performs focus adjustment for thephysical object based on the ranging values, and comprises subjectingranging values of the plurality of detection areas to statisticalprocessing, calculating representative values that contain statisticaldispersion based on the statistical processing, and estimating movementstate of the physical object based on time-series change in therepresentative values, predicting focal position regarding the physicalobject based on the movement state that has been estimated, andperforming focus adjustment based on the focal position that has beenpredicted.

A non-transitory computer-readable medium of a third aspect of thepresent invention, storing a processor executable code, which whenexecuted by at least one processor, the processor being arranged in afocus adjustment device that is provided with a plurality of detectionareas in an imaging region formed by an optical system, and that detectsranging values of a physical object in the detection areas to performfocus adjustment for the physical object, the focus adjustment methodcomprising subjecting ranging values of the plurality of detection areasto statistical processing, calculating representative values thatcontains statistical dispersion based on the statistical processing, andestimating movement state of the physical object based on time-serieschange in the representative values, predicting focal position regardingthe physical object based on the movement state that has been estimated,and performing focus adjustment based on the focal position that hasbeen predicted.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram mainly showing the electrical structure of acamera of one embodiment of the present invention.

FIG. 2 is a block diagram showing detail of an AF calculation section ofa camera of one embodiment of the present invention.

FIG. 3 is a flowchart showing AF operation of the camera of oneembodiment of the present invention.

FIG. 4 is a flowchart showing operation of statistical processing of thecamera of one embodiment of the present invention.

FIG. 5A and FIG. 5B are flowcharts showing operation of focal positionprediction processing of the camera of one embodiment of the presentinvention.

FIG. 6 is a drawing for describing division of data relating to rangingof target ranging areas into regions, in the camera of one embodiment ofthe present invention.

FIG. 7 is a drawing for describing the fact that a small number ofregions at the close-up end are removed, in the camera of one embodimentof one embodiment of the present invention.

FIG. 8 is a drawing for describing the fact that regions at the infinityend are excluded, using history information, in the camera of oneembodiment of one embodiment of the present invention.

FIG. 9 is a drawing for describing division of data relating to rangingof target ranging areas, using quartiles, in the camera of oneembodiment of the present invention.

FIG. 10 is a table showing frequency distribution of interquartilerange, in the camera of one embodiment of the present invention.

FIG. 11 is a graph showing chronological change of each difference valuein a case where data relating to range has been divided intointerquartile ranges, in the camera of one embodiment of the presentinvention.

FIG. 12 is a drawing showing a comparison of each difference value and athreshold value, in a case where data relating to range has been dividedinto interquartile ranges, in the camera of one embodiment of thepresent invention.

FIG. 13 is one example showing results of adopting quartiles, in thecamera of one embodiment of the present invention.

FIG. 14 is a drawing showing an example where the present invention hasbeen applied to a cell culture device.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

One embodiment of the present invention will be described in thefollowing. The present invention can be applied to a focus adjustmentdevice if the device has a plurality of detection areas (ranging areas)in an imaging region, repeatedly calculates ranging values of an objectin the plurality of detection areas to acquire ranging values, andperforms focus adjustment for the object. Here, description will begiven of an example where the present invention has been applied to adigital camera (called a “camera”), as one embodiment.

This camera has an imaging section, with a subject image, which is anobject, being converted to image data by this imaging section, and thesubject image being subjected to live view display on a display sectionarranged on the rear surface etc. of the camera body based on thisconverted image data. A photographer determines composition and photoopportunity by looking at the live view display. At the time of arelease operation image data is stored in a storage medium. Image datathat has been stored in the storage medium can be subjected to playbackdisplay on the display section if playback mode is selected.

Also, it is possible to detect ranging values for each of a plurality ofranging areas using pixel data that is output by the imaging section ofthis camera (refer, for example, to S1 in FIG. 3 ). A statistical valuerepresenting dispersion in ranging values of a plurality of rangingareas is then calculated, and representative values that include thestatistical value representing this dispersion are obtained (refer, forexample, to S5 to S11 in FIG. 3 ). Movement of the subject is estimatedbased on time series change in the representative values (refer, forexample, to S15 in FIG. 3 ). It should be noted that the imaging sectionreads out pixel data at time intervals corresponding to a specifiedframe rate, and outputs the pixel data to the AF calculation section.

As a statistical method for obtaining the above-described statisticalvalues representing dispersion, in this embodiment quartiles are used.As the representative values including the statistical valuerepresenting dispersion, a first quartile (Q1), second quartile (Q2),third quartile (Q3), and interquartile range (Q1 to Q3) etc. are used(refer, for example to S11 in FIG. 3 , and to FIG. 9 , and in thefollowing Q1, Q2, and Q3 will be called quartiles). These quartiles arecalculated based on ranging values of a plurality of ranging areas forevery frame that has been imaged. If quartiles have been calculated forevery frame, differences between associated quartiles Q1, Q2 and Q3between frames are calculated (refer, for example, to FIG. 11 ).Further, movement state of the subject is estimated based on change indifferences between associated quartiles Q1, Q2, and Q3 across aplurality of frames (refer, for example, to S15 in FIG. 3 , and to FIG.12 ). This estimated subject movement state is reflected at the time ofmoving body estimation computation (refer, for example, to S19 in FIG. 3, and to FIG. 5A and FIG. 5B). For example, subject movement state isreflected in the order of the moving body prediction equation and numberof items of history information etc. when performing moving bodyestimation computation.

One embodiment of the present invention will be described in thefollowing using the drawings. FIG. 1 is a block diagram mainly showingthe electrical structure of the camera of this embodiment. The camerarelating to this embodiment comprises an interchangeable lens 10 and acamera body 20. With this embodiment, the interchangeable lens 10 and acamera body 20 are configured separately, but they may also beconstructed integrally, like a general compact camera. Also, the cameraof this embodiment receives light of a subject image that has beensubjected to pupil division by a photographing lens, that includes afocus lens, using an image sensor, subjects the subject image tophotoelectric conversion to generate image data, and performs a focusadjustment operation based on the image data.

A photographing lens 11 is arranged inside the interchangeable lens 10.The photographing lens 11 has a plurality of optical lenses, including afocus lens, and forms an optical image of a subject S. An aperture isarranged within the interchangeable lens 10, in the optical path of thephotographing lens 11, and an aperture opening detection section fordetecting opening of this aperture is also provided within theinterchangeable lens 10. An actuator 12 and a lens control section 13are also provided inside the interchangeable lens 10.

The lens control section 13 is one or a plurality of processors having aCPU (Central Processing Unit), peripheral circuits, and a memory thatstores programs etc. The lens control section 13 receives a defocusamount (or corresponding lens pulse value) from an AF calculationsection 23 inside the camera body 20, and performs control of theactuator 12 based on these items of information. The actuator 12 movesthe focus lens within the photographing lens 11 in the optical axisdirection to perform focusing. It should be noted that position of thefocus lens is detected using a lens position detection section (notillustrated), and lens position is transmitted by means of acommunication section (not illustrated) to the camera body 20.

An image sensor 21, image processing section 22, AF calculation section23, and storage section 24 are provided within the camera body 20. Theimage sensor 21 is arranged on the optical axis of the photographinglens 11, close to a position where a subject image is formed.

The image sensor 21 is provided with a plurality of pixels, and eachpixel has a photoelectric conversion section for converting the subjectimage (optical image) to an electrical signal. Also, there are two typesof the above-described pixels, namely image pixels and focus detectionpixels. Incident direction of light flux that is incident on the focusdetection pixels (also called phase difference detection pixels) isrestricted. On the other hand, light flux that is incident on the imagepixels is restricted less than for the focus detection pixels. Thesepixels are arranged two-dimensionally. Also, pixels of the image sensorare arranged so as to form a plurality of focus detection pixel lines(focus detection areas).

As the previously described focus detection pixels, focus detectionpixels (for example, left opening focus detection pixels) are providedto receive light of one direction, resulting from having divided lightthat is incident from the photographing lens 11 into two directions(left and right, or up and down) by pupil division, and focus detectionpixels (for example, right opening focus detection pixels) are providedto receive light of the other direction. Pairs are then formed of focusdetection pixels receiving light of the one direction and of the otherdirection. For example, a plurality of pixel data of a plurality of leftand right opening focus detection pixels constitute pairs for phasedifference in a left-right direction. The image sensor 21 functions asan image sensor that has a plurality of pixel sections that subjectlight flux from the subject, that has passed through the photographinglens, to pupil-division, and that are constituted by pairs of pixelsthat receive the light that has been subjected to pupil-division, andoutputs pairs of pixel signal lines corresponding to pupil divisionresulting from photoelectric conversion of the plurality of pixelsections.

Restriction of the incident direction of light flux that is incident onthe pixels may also be realized using focus detection pixels, and apositional relationship between micro lens that are arranged on thesurface of those focus detection pixels, and the photographing lens 11.Specifically, it is possible to restrict incident direction of the lightflux, by displacing position of the focus detection pixels with respectto the optical axis of a micro lens. The image sensor 21 that adoptsthis type of structure functions as an image sensor that has a pluralityof pixel sections made up of a plurality of pixels that are paired inaccordance with a single micro lens, and that outputs a plurality ofpixel signals in accordance with pupil-division resulting fromphotoelectric conversion by the plurality of pixel sections.

The image sensor 21 outputs pixel data (pixel values) that has beenoutput from the focus detection pixels and the image pixels to the imageprocessing section 22 and the AF calculation section 23. The imageprocessing section 22 has an image processing circuit, is input withpixel data from the imaging pixels, among the pixel data, and performsimage processing for a live view display image and for a storage image.The image processing section 22 also outputs image data, that has beenprocessed for storage, to the storage section 24. The storage section 24is an electrically rewritable nonvolatile memory, to which image datafor storage is input and stored. The image processing section 22 alsodetects a face of a subject using pixel data and outputs a centralcoordinate position of this face, and detects organs such as eyes of theface and outputs a specified coordinate position of this organ (refer tothe face detection section 22 a which will be described later). Theimage processing section 22 also performs subject tracking using pixeldata (refer to the tracking section 22 b which will be described later).

The AF calculation section 23 is one or a plurality of processors havinga control circuit such as a CPU (Central Processing Unit), peripheralcircuits, and a memory that stores programs etc. The AF calculationsection 23 is input with pixel data from the focus detection pixels,among pixel data, and performs AF calculation based on phase differencedetection. At the time of AF calculation, a ranging area (focusdetection area) corresponding to position of the focus detection pixelsis set based on central coordinate position and specified coordinateposition that have been acquired from the image processing section 22.The AF calculation section 23 then calculates defocus amount andcontrast evaluation value for this ranging area that has been set. Thefocus lens within the photographing lens 11 is driven to an in focusposition based on this defocus amount and contrast evaluation value thathave been calculated. The AF calculation section 23 acts in cooperationwith the lens control section 13 to function as a control section thatperforms focus adjustment based on focal position that has beenpredicted. The detailed structure of the AF calculation section 23 willbe described later using FIG. 2 .

Next, details of the AF calculation section 23 will be described usingFIG. 2 . The AF calculation section 23 comprises one or a plurality ofprocessors, as was described previously, and has functions as astatistical processing section, movement state estimating section,prediction section, and control section. Specifically, each of theabove-described sections may be part of the processor. Also, the AFcalculation section 23 has an AF ranging point setting section 33, phasedifference pixel generating section 34, first AF calculation section 35,second AF calculation section 36, and focal position prediction section37, and each of the sections may be realized by hardware circuits withinthe processor, and programs executed by a CPU.

The pixel data 21 a in FIG. 2 is pixel data (pixel values) that has beenoutput from the image sensor 21, and is temporarily stored in SDRAM(Synchronous Dynamic Random Access Memory) or the like. As was describedpreviously, imaging pixels and focus detection pixels (phase differencedetection pixels) are provided in the image sensor 21, with the imageprocessing section 22 processing mainly pixel data from the imagingpixels, and the AF calculation section 23 processing pixel data from thefocus detection pixels.

A face detection section 22 a having a face detection circuit isprovided within the image processing section 22. This face detectionsection 22 a determines whether or not there is a face within a subjectimage based on pixel data of the imaging pixel from the image sensor 21.If the result of this determination is that a face is contained withinthe subject image, the face detection section 22 a detects position(central coordinate position) and size etc. of the face. Further,detection of organs such as the right eye, left eye, nose etc. is alsoperformed, and specified coordinate position of those organs is alsodetected. Central coordinates and/or specified coordinate positions thathave been detected by the face detection section 22 a are output to anAF ranging point setting section 33 within the AF calculation section23.

A tracking section 22 b having a tracking circuit is also providedwithin the image processing section 22. This tracking section 22 bperforms tracking of a subject based on pixel data of imaging pixelsfrom the image sensor 21. The tracking section 22 b compares pixel data,every time pixel data is output from the image sensor 21, based on, forexample, position of a face that has been detected by the face detectionsection 22 a, position of the subject that has been designated by thephotographer, or position of a subject that is at the closest distancewithin a plurality of ranging areas. Based on the results of thiscomparison, the tracking section 22 b detects where the same subject hasmoved to within the imaging region (photographing screen), and in thisway performs tracking. Central coordinate and/or specified coordinatepositions within a tracking target that has been detected by thetracking section 22 b are output to an AF ranging point setting section33 within the AF calculation section 23.

The AF ranging point setting section 33 that is provided within the AFcalculation section 23 sets AF ranging point (ranging area)corresponding to the central coordinate position and/or specifiedcoordinate position that has been detected by the face detection section22 a or the tracking section 22 b, based on that central coordinateposition and/or specified coordinate position. A plurality of rangingpoints are associated with imaging regions (photographing screen) of theimage sensor 21 beforehand, and ranging points at the central coordinateposition and/or specified coordinate position, or ranging points thatare close to these coordinate positions, among the plurality of rangingpoints, are set, and central coordinates of each ranging point that hasbeen set are output to the phase difference pixel generating section 34and the second AF calculation section 36. It should be noted thatranging points can also be set manually by the user.

A focus detection pixel (phase difference pixel) generating section 34is input with phase difference pixel data for focus detection pixellines, within the pixel data 21 a. The focus detection pixel generatingsection 34 is also input with central coordinates of ranging (FOCUSdetection) areas etc. from the AF ranging point setting section 33, andgenerates phase difference pixel data lines corresponding to, or closeto, an AF ranging point that has been set from among phase differencepixel data. The generation of phase difference pixel data lines by thefocus detection pixel generating section 34 is not limited to only areasthat belong to central coordinates of a ranging area that has beenoutput from the AF ranging point setting section 33, and phasedifference pixel data lines are generated for a plurality of areasaround this area. It should be noted that AF ranging point is notlimited to a ranging point that has been determined by the facedetection section 22 a and/or the tracking section 22 b, and may also beset manually by the photographer, and may be set using an inferencemodel that has been learned by deep learning. Further, the AF rangingpoint is not limited to a local ranging point, and may also be a widearea of the screen and may be all ranging points within the screen. Thisphase difference pixel data that has been generated is output to thefirst AF calculation section 35.

The first AF calculation section 35 has a defocus amount calculationsection 35 a and a reliability evaluation section 35 b. The defocusamount calculation section 35 a calculates phase difference using phasedifference pixel data lines for left and right openings. The defocusamount calculation section 35 a calculates defocus amount of the focuslens based on a known phase difference method. Specifically, the phasedifference pixel generating section 34 outputs a phase difference pixeldata line that has been output by left opening phase difference pixels,and a phase difference pixel data line that has been output by rightopening phase difference pixels. The defocus amount calculation section35 a then calculates a degree of correlation while shifting the twophase difference pixel data lines, and calculates defocus amount of thefocus lens based on a shift amount where this degree of correlationbecomes maximum. Refer, for example, to FIG. 5 to FIG. 7 of Japanesepatent laid open number 2018-097253 (US patent application publicationnumber US 2018/0176453). US Patent Application Publication No. US2018/0176453 is incorporated herein by reference.

The reliability evaluation section 35 b evaluates reliability of thedefocus amount data has been calculated by the defocus amountcalculation section 35 a. Evaluation of reliability of the defocusamount may use a known method. For example, reliability of the defocusamount may be evaluated based on gradient of degree of correlation inthe vicinity of shift amount where the degree of correlation becomesmaximum (refer, for example, to FIG. 7 in Japanese patent laid opennumber 2018-097253 (US patent application publication number US2018/0176453).

The first AF calculation section 35 outputs defocus amounts for eachranging area, that have been calculated by the defocus amountcalculation section 35 a, and an evaluation value for reliability thathas been calculated by the reliability evaluation section 35 b, to thesecond AF calculation section 36.

The second AF calculation section 36 comprises a statistical processingsection 36 a, movement state estimating section 36 b, and ranging areaselection section 36 c. The statistical processing section 36 a performsstatistical processing on a lens pulse value resulting from havingconverted defocus amount for each ranging area that was calculated inthe defocus amount calculation section 35 a to a focus lens positionwhere focus is achieved. As statistical processing, in this embodimentquartiles are obtained (refer, for example, to S11 in FIG. 3 and to FIG.9 ). A quartile is a partition value when dividing a number of datapoints into four parts, with data arranged from smallest to largest.Quartiles will be described later using FIG. 9 . Also, before obtainingquartiles, in this embodiment a plurality of lens pulse values that havebeen calculated are divided into a plurality of lens pulse regions(refer to S5 in FIG. 3 , and to FIG. 6 , which will be described later),and among the lens pulse regions that have been divided, lens pulseregions for the close-up end and the infinity end are excluded (refer toS7 and S9 in FIG. 3 , and to FIG. 7 FIG. 8 , which will be describedlater). Quartiles are obtained using this lens pulse region division,and data of lens pulse values that have been subjected to close-up endand infinity end exclusion processing. Operation of the statisticalprocessing performed by the statistical processing section 36 a will bedescribed later using FIG. 4 .

The statistical processing section 36 a functions as a statisticalprocessing section that performs statistical processing on rangingvalues of a plurality of detection areas (refer, for example, to S5 toS11 in FIG. 3 , and to FIG. 4 and FIG. 6 to FIG. 9 ). Theabove-described statistical processing section uses quartiles. Thestatistical processing section calculates standard deviation and/oraverage values. With this embodiment, the statistical processing section36 a calculates quartiles, but the invention is not limited tocalculating quartiles, and standard deviation and average values may becalculated, and movement of the subject estimated using representativevalues (for example, average value±standard deviation) that includestatistical values representing dispersion.

The movement state estimating section 36 b estimates movement of thesubject. When estimating this subject movement, the movement stateestimating section 36 b uses quartiles that have been calculated by thestatistical processing section 36 a. The statistical processing section36 a calculates quartiles every time pixel data for one frame is output,and so the movement state estimating section 36 b estimates subjectmovement based on change in differences between quartiles (Q1, Q2, Q3)for the previous frame and the current frame (refer, for example, to S15in FIG. 3 ). This estimation is performed when reliability of thequartiles is high (refer to S13 in FIG. 3 ). Estimation of subjectmovement will be described later using FIG. 11 to FIG. 13 .

The movement state estimating section 36 b functions as a movement stateestimating section that calculates representative values containingstatistical dispersion based on statistical processing, and estimatesmovement state of an object based on time series change inrepresentative values (refer, to S15 in FIG. 3 , and to FIG. 11 to FIG.13 ). In this embodiment, quartiles such as 1st to 3rd quartiles areused as representative values, but if there are values resulting fromhaving subjected ranging values of a plurality of detection areas tostatistical processing, it is possible to use representative values thatinclude values other than quartiles, such as, for example, averagevalues and standard deviation. That is, a representative value may be avalue that includes average value and/or standard deviation. Forexample, statistical dispersion may be standard deviation and/or meandeviation, and representative values that include statistical dispersionmay be average value±standard deviation and/or average value±meandeviation.

The above mentioned movement state estimating section estimates movementdirection of an object based on time series change in representativevalues (refer, for example, to FIG. 5A, FIG. 5B, and FIG. 12 ). In thisembodiment, quartiles such as 1st to 3rd quartiles (Q1, Q2, Q3) are usedas representative values that contain statistical dispersion. Differencevalues for different times are used as time series change inrepresentative values, and movement direction is estimated based onthese difference values (refer, for example, to FIG. 12 ). The movementstate estimating section estimates movement speed of an object based ontime series change in representative value (refer, for example, to FIG.12 ). In this embodiment, quartiles such as 1st to 3rd quartiles (Q1,Q2, Q3) are used as representative values that contain statisticaldispersion. Difference values for different times are used as timeseries change in representative values, and movement speed of a subjectis estimated by comparing these difference values with a specifiedthreshold value (refer, for example, to FIG. 12 ).

The above mentioned movement state estimating section determinesreliability based on time series change in representative values, and ifit is determined that there is reliability, estimates and outputsmovement state of an object (refer, for example, to S13 in FIG. 3 , andto FIG. 10 ). The movement state estimating section makes arepresentative value the first quartile, or the second quartile, or thethird quartile. The movement state estimating section determinesreliability based on time series change in interquartile range, and ifit is determined that there is reliability, estimates and outputsmovement state of an object (refer, for example, to S13 and S15 in FIG.3 , and to FIG. 10 to FIG. 13 ). The movement state estimating sectiondetermines reliability based on degree of dispersion in interquartilerange, size of interquartile range, or number of effective rangingvalues contained in the interquartile range (refer, for example, to S13in FIG. 3 ).

The ranging area selection section 36 c selects ranging areas to be usedat the time of focus adjustment using the focus lens, based on lenspulse values resulting from having converted defocus amounts for eachranging area that were calculated by the defocus amount calculationsection 35 a (refer, for example, to S17 in FIG. 3 ). Selection ofranging areas is performed using a known method, such as areas that haveclosest range data among the plurality of ranging result, areas thathave been selected manually by the photographer, areas in which a faceis been detected, and areas that have been selected based onconventional moving body prediction etc. It should be noted that theranging area selection section 36 c is not limited to data of a singleranging area, and data resulting from having processed data of aplurality of related ranging areas may also be output.

The second AF calculation section 36 outputs lens pulse values resultingfrom having converted the focus amounts for ranging areas that wereselected by the ranging area selection section 36 c, and subjectmovement that has been estimated by the movement state estimatingsection 36 b, to the focal position prediction section 37. The focalposition prediction section 37 calculates (predicts) position of thefocus lens where focus will be achieved, at the time of actual shooting,based on history of lens pulse values resulting from having converteddefocus amounts for selection areas. Based on this predicted value, alens command value (lens pulse value) is transmitted to the lens controlsection 13 within the interchangeable lens 10. The focal positionprediction section 37 can perform moving body prediction to predictposition of the focus lens where focus is achieved at the time of actualshooting using a known method. Specifically, known moving bodyprediction is a method of calculating position of the focus lens wherefocus is achieved at the time of actual shooting using history ofposition of the focus lens (lens pulse values) and a specified movingbody estimation computation equation.

Also, the focal position prediction section 37 of this embodiment canpredict position of the focus lens where focus will be achieved at thetime of actual shooting, using movement state of the subject that hasbeen estimated by the movement state estimating section 36 b. Thisprediction will be described later using FIG. 5A and FIG. 5B. The focalposition prediction section 37 functions as a prediction section thatpredicts focal position relating to an object based on movement state ofa subject that has been estimated (refer to S19 in FIG. 3 , and to FIG.5A and FIG. 5B).

Next, AF operation of this embodiment will be described using theflowchart shown in FIG. 3 . This flow is executed by the CPU etc. thatis provided within the AF calculation section 23 controlling eachsection shown in FIG. 1 and FIG. 2 based on programs that have beenstored in the nonvolatile memory.

If the photographer performs a shooting instruction operation, such asby pressing down the release button halfway (1st release), and the imagesensor 21 acquires image data for one frame, the AF operation shown inFIG. 3 is commenced. If AF operation is commenced, first, rangingcalculation is performed (S1). Here, center coordinate/range is set foreach ranging area that has been set by the AF ranging point settingsection 33, based on central coordinates/specified coordinates that havebeen output from the image processing section 22. The phase differencepixel generating section 34 generates left and right opening AF pixeldata lines based on the settings, and the defocus amount calculationsection 35 a within the first AF calculation section 35 calculatesdefocus amount of each ranging area using the AF pixel data lines.Calculation of defocus amount is performed using a known phasedifference AF method (refer, for example, to FIG. 5 to FIG. 7 ofJapanese patent laid open number 2018-097253 (US patent applicationpublication number US 2019/0176453). It should be noted that, as will bedescribed later, if the processing of step S19 is completed, and ifoperation of the release button is maintained, processing returns tostep S1 and ranging calculation is repeated.

If ranging calculation has been performed, next, reliabilitydetermination and exclusion processing are performed (S3). Here, thereliability evaluation section 35 b evaluates reliability of the defocusamount that has been calculated in step S1. Reliability is evaluated,for example, based on inclination of degree of correlation in thevicinity of shift amount where the degree of correlation, that wasobtained at the time of calculation of defocus amount, becomes maximum(refer, for example, to FIG. 7 in Japanese patent laid open number2018-097253 (US patent application publication number US 2018/0176453)).Data from ranging areas for which it has been determined that evaluationvalue for reliability is lower than a predetermined value, and that aretherefore not reliable, is excluded, and that data is not used insubsequent processing.

If reliability determination and exclusion processing have beenperformed in step S3, the statistical processing section 36 a performsstatistical processing in steps S5 to S11 (detailed operation of thisstatistical processing will be described later using FIG. 4 ). In thisstatistical processing, quartiles are calculated, and in thiscalculation of quartiles region division is first performed (S5). Inthis embodiment, in order to reduce a number of lens pulse values forwhich calculation of quartiles is performed, and thus reduce processingtime, lens pulse values that are estimated to correspond to unwantedsubjects other than the main subject (crossing subjects and backgroundetc.) are excluded. To do this there is division into a plurality oflens pulse regions by determination from distribution of lens pulsevalues of groups of target ranging areas, and threshold values.Respective defocus amounts for ranging areas constituting targets, suchas ranging areas that have been set manually by the photographer, orranging areas corresponding to tracking that has been automatically setinside the camera, ranging area groups contained in recognized areasresulting from face detection and deep learning etc., and all rangingareas within the photographing screen, are then converted to lens pulsevalues. Lens pulse values are values resulting from having converteddefocus amounts to positions of the focus lens corresponding to infocus, as was described previously. Once there has been conversion tolens pulse value, all lens pulse values arranged in ascending order. Ifthe lens pulse values have been arranged, next, differences betweenadjacent lens pulse values are calculated. A specified number of higherranking larger difference values are then compared with a thresholdvalue, and once a difference value becomes larger than the thresholdvalue, the lens pulse value corresponding to that difference value isset as a boundary line of a lens pulse region.

Region division will be specifically described using FIG. 6 . Thevertical axis in FIG. 6 is lens pulse value, with the lower part of thevertical axis being infinity and the upper part of the axis being theclose-up end. White circles within the frame Vm are values resultingfrom having converted defocus amounts for each ranging area, that havebeen generated by ranging (focus detection) once (a frame), to lenspulses, and these lens pulse values (values of target lens pulsepositions) are plotted. Once the lens pulse values have been plotted,next, difference values between two adjacent lens pulse values arecalculated (this value is called difference pulse value).

If difference pulse values have been obtained, the difference pulsevalues are compared with a specified threshold value, in order of sizeof the difference pulse values, and it is determined whether thedifference pulse value is greater than the threshold value. Thisdetermination is performed for only a specified number of differencepulse values, starting from the largest, in order to shorten processingtime. With the example shown in FIG. 6 , determination is performed forthe difference pulse values that are ranked 1st to 5th from the largest,and determination is not performed for those values ranked 6th and 7th.In FIG. 6 , the difference pulse value that are ranked 1st to 4th aregreater than the threshold value and so are made a boundary (line),while the 5th ranked difference pulse value is less than the thresholdvalue and is not made a boundary (line).

Based on the result of this determination a boundary line is drawnbetween two lens pulse values that have difference pulse values that areapart by more than the threshold value, thus dividing lens pulse valuesof each ranging area into a plurality of lens pulse regions. With theexample shown in FIG. 6 there is division into five regions, namelyregion 0 to region 4. Since lens pulse values within each lens pulseregion have a difference pulse value that is within a specifiedthreshold value, distances to an object corresponding to lens pulsevalues within the same region are within a specified range, and it isdetermined that it is the same object.

If the division into lens pulse regions has been performed in step S5,close-up end region exclusion is performed next (S7). After lens pulseregion division, and after having excluded regions other than lens pulseregions relating to the main subject, quartiles are applied. This isbecause if information for other than the main subject is also included,reliability of results to which quartiles have been applied will becomelower. With regard to the close-up end, lens pulse regions locatedcloser to the lens pulse values (target lens pulse positions) that havebeen set using history information of quartiles are specified. Then, ina case where a number of areas (number of lens pulse values) that areincluded in this lens pulse region is a small number that is less than aspecified number, it is determined to be a lens pulse regioncorresponding to an unwanted subject such as a crossing subject, or alens pulse region that includes a ranging error, and the lens pulseregion is excluded.

FIG. 7 is one example of exclusion of a small number of close-up endlens pulse regions, and exclusion of a small number of close-up end lenspulse regions will be described using FIG. 7 . In FIG. 7 also, thevertical axis is lens pulse value, with the lower part of the verticalaxis being infinity and the upper part of the axis being the close-upend, and the horizontal axis represents time. Similarly to FIG. 6 ,white circles within the frame Vm1 plot lens pulse value of each rangingarea that has been calculated in the current frame. With the exampleshown in FIG. 7 lens pulse values of each ranging area are divided intofour, namely lens pulse regions 0 to 3.

In FIG. 7 , Qa0 is a box plot (or box-and-whisker plot) showingquartiles that have been calculated in the previous frame (quartileswill be described later using FIG. 9 ). A maximum value Max of thisquartile is compared with each lens pulse value that was calculated inthis frame (refer to within frame Vm1), and with the example shown inFIG. 7 the lens pulse values belonging to lens pulse regions 1 and 0 areboth greater than the maximum value Max. Of these, the number of lenspulse values belonging to lens pulse region 1 is greater than a fixednumber (for example, three), but the number of lens pulse valuesbelonging to lens pulse region 0 is not greater than the fixed number.With the number of lens pulse values contained in the lens pulse region0 being less than the fixed number, it is determined that a subjectcorresponding to the lens pulse values of this lens pulse region 0 is anunwanted subject such as a crossing subject, or that there is a highpossibility of the lens pulse values including a large ranging error. Asa result of this, the lens pulse values belonging to lens pulse region 0are excluded when calculating quartiles for the current frame.

If close-up end lens pulse region exclusion has been performed in stepS7, then next, infinity end lens pulse region exclusion is performed(S9). Here, exclusion is performed by comparing maximum lens pulse valuein the current frame with history information of quartiles (specifiednumber of previous frames). Also, results of estimating subject movementestimated up to the previous frame (refer to step S15 which will bedescribed later) are also referenced, and if there are lens pulseregions detected conclusively away from the infinity end they areexcluded. It should be noted that at a time corresponding to a specifiednumber of frames the image sensor 21 performs readout of pixel data forone screen, and the “current frame” is pixel data for one screen thathas been read out immediately after.

Also, as infinity end lens pulse region exclusion, regions having lenspulse values that are smaller than a value resulting from havingsubtracted a fixed value from the maximum value of lens pulse value forthe current frame are excluded. Further, similarly to the close-up end,if only a few lens pulse values are contained in a lens pulse region,that lens pulse region is excluded. By performing these processes,background subjects etc. of the main subject are separated and excluded.

FIG. 8 is one example of exclusion of infinity end lens pulse regions,and exclusion of infinity end lens pulse regions will be described usingFIG. 8 . In FIG. 8 also, the vertical axis is lens pulse value, with thelower part of the vertical axis being infinity and the upper part of theaxis being the close-up end. Also, the horizontal axis in FIG. 8represents lapse of time. White circles within frame Vm2 are examples oflens pulse values for each ranging area, in the current frame, similarlyto FIG. 6 , and there is division into lens pulse regions 0 and 1.

In FIG. 8 , box plots Qa1 to Qa5 show history of quartiles correspondingto previous frames, with box plot Qa1 being quartiles corresponding tothe previous frame, and box plot Qa2 being quartiles for the framebefore that. Quartiles in a frame three frames before are invalid (shownby Qa3, specifically, Qa3 is a quartile invalid frame), and quartilesfor 4 frames before and 5 frames before are the box plots Qa4 and Qa5,respectively. With the example shown in FIG. 8 , a maximum value of lenspulse value within the current frame Vm2 is smaller than a minimum valueLstMin of quartiles for the previous frame Qa1 (at the infinity endside), and so lens pulse regions 0 and 1 are excluded (refer to S27 andS29 in FIG. 4 ). In this way, if valid lens pulse values do not exist inthe current frame it is determined to be a state where a main subjecthas been out of the target ranging areas. In this type of state, lock onprocessing that does not move the focus lens is performed, and anoperation is performed to await generation of valid lens pulse valuesfor the next frame. As a result, movement of the focus lens to the wrongposition is prevented.

In FIG. 8 , white circles within frame Vm3 are another example of lenspulse values for each ranging area, in the current frame, similarly toFIG. 6 , and there is division into lens pulse regions 0 and 1. Withthese other examples, lens pulse region 1 is positioned more toward theinfinity end than a value resulting from having subtracted a particularfixed value Tha from the maximum value PreMax for the current frame, andthis lens pulse region 1 is excluded (refer to S31 and S33 in FIG. 4 ).In this way, lens pulse values having a difference from the maximum lenspulse value of the current frame that is larger than Tha are determinedto be unnecessary background, and are excluded.

In this way, in steps S5 to S9, there is division into respective lenspulse regions based on lens pulse values that have been acquired foreach ranging area, and for each lens pulse region, a lens pulse regionfor the close-up end that has been determined to contain only a smallnumber of lens pulse values is excluded. Lens pulse regions at theinfinity end are also excluded by comparing history informationquartiles and threshold values that have been set in advance. As aresult of these processes it is possible to exclude lens pulse valuescorresponding to other than the main subject with good precision, and itis possible to retain only lens pulse values corresponding to the mainsubject. Next, quartiles for the current frame are calculated using thelens pulse values corresponding to this main subject (S11).

In this step S11, quartiles are applied by arranging a plurality of lenspulse values that have been calculated in ascending order. A quartile isa statistical method in which a center value Q2 (second quartile) in anascending array of all data is obtained, data is divided into tworegions with this value as a boundary, center values Q1 (first quartile)and Q3 (third quartile) within each of the respective regions that havebeen divided into two are obtained, and there is further division intoregions with those two center values as boundaries, to give threeboundaries and four sections. Quartiles can be illustrated as box plots,and an image of a box plot Qa is shown in FIG. 9 . In the box plot Qa,the nine white circles represent lens pulse values that have beenacquired in each ranging area (as one example, 9 points) similarly toFIG. 6 etc.

Size of a box is determined using values Q1 to Q3. The size of the box,specifically the interquartile range IQR, is Q3−Q1 (a range from valueQ3 to value Q1). Also, lengths of whiskers Wh1 and Wh2 are determinedbased on the box size (IQR) and a predetermined threshold value Th,using Wh1=IQR×Th, Wh2=IQR×Th. Also, a minimum value of lens pulse valuesthat exists within the length of the whiskers Wh1 and Wh2 is set as Min(minimum value used in quartiles), while a maximum value of lens pulsevalues that exists within the length of the whiskers Wh1 and Wh2 is setas Max (maximum value used in quartiles), and between the maximum valueand minimum value (Max-Min) in quartiles is made range R. Lens pulsevalues that exist outside the lengths of the whiskers Wh1 and Wh2 areexcluded as outlier values Ot.

Values of Min (minimum value within the range, infinity side), Q1 (25%position), Q2 (50% position), Q3 (75% position), and Max (maximum valuewithin the range, close-up end), and information such as current lenspulse value, are saved as time series data, and movement state(movement) of the subject (object) is estimate using history informationof those values. In order to divide into the three boundaries and foursections, the number of lens pulse values (number of correspondingranging areas) is desirably greater than a fixed number (a minimum beingmore than 5).

Once the quartiles have been calculated in step S11, reliability of thequartiles is next determined (S13). Here, the movement state estimatingsection 36 b determines reliability of the quartiles that werecalculated in step S11. First, in order to measure degree of dispersionin the quartiles that have been calculated, history information of aninterquartile range corresponding to a plurality of frames that wereacquired in a given fixed period is made into a frequency distributiontable (histogram) such as shown in FIG. 10 , for example. It should benoted that the statistical processing section 36 a may also determinereliability of the quartiles.

In FIG. 10 , values for a plurality of interquartile ranges (IQR=Q3−Q1:refer to FIG. 9 ) corresponding to a plurality of frames that wereacquired in a fixed period, are compared with ranges of classes thathave been set beforehand, and corresponding classes are counted toobtain a number of times (frequency). Next, a relative frequency(=number of times in that class/total number of times (total in FIG. 10) is calculated for every class. In FIG. 10 a class is set withreference to F6, in units resulting from having converted allowabledefocus amount F6 to lens pulse value, with F being aperture value and δbeing permissible circle of confusion. For example, if a total amount ofrelative frequency for the first and second ranking classes that has themost frequent value is greater than a threshold value, dispersion ininterquartile range that was acquired in a fixed period is small, thatis, dispersion in the quartile is small, and it is determined thatreliability is high. With the example shown in FIG. 10 , a total ofrelative frequency for two classes, namely class a˜b (first mostfrequent value) of interquartile range and class b˜c (second mostfrequent value) becomes 0.8 (0.4+0.4). The total of this relativefrequency is compared with a threshold value (for example, 0.7), andsince the total for relative frequency is larger than the thresholdvalue it can be determined that reliability of the quartiles is high.

Further, by comparing a value for interquartile range IQR with athreshold value for every frame, if the interquartile range is less thanthe threshold value and the range is sufficiently narrow it is assumedthat lens pulse values (ranging areas) that have applied quartiles onlycorrespond to the main subject, and the quartiles for that frame can bedetermined to have high reliability. If the interquartile range IQR islarger than the threshold value then dispersion in lens pulse values islarge, it is assumed to be a case where a background is also included aswell as the main subject, or a case where the main subject itself hasdepth, and it can be determined to be a state where reliability is low.

Also, values of the range R (max-min) based on the quartiles (FIG. 9 )tend to have large dispersion compared to the interquartile range IQR.They can therefore be used in reliability determination by looking at acorrelation relationship with a range R and values of the interquartilerange IQR. For example, for a class of the first or second most frequentvalues of interquartile range IQR (FIG. 10 : a˜b or b˜c), if they becomea class where the most frequent range value of R (b˜c) is larger, or aclass that is the same, it is determined that reliability of thequartile is high.

Besides the above described method, reliability of both quartiles andresults of applying those quartiles may be determined collaborativelyusing numbers of valid lens pulse values (ranging areas) contained inthe interquartile range of each frame, and/or information such asproportion of number of frames for which quartiles can be applied, amonga plurality of continuous frames. If reliability of a quartile is low,quartiles for that frame are not used in estimation of subject movement(refer, for example, to Qa3 in FIG. 8 ).

Once reliability of the quartiles has been determined in step S13,subject movement is next estimated (S15). When predicting focal positionin step S19, subject movement is taken into consideration. In this stepS15, therefore, the movement state estimating section 36 b estimatessubject movement using the quartiles that were calculated in step S11.

In step S15, in order to estimate subject movement in a short period,the movement state estimating section 36 b uses quartiles Q1˜Q3 of twocontinuous frames. On the other hand, since minimum value Min andmaximum value Max used in the quartiles tend to have large dispersioncompared to Q1˜Q3, they are not used. Differences between respectiveQ1˜Q3 between the current frame and the previous frame are calculated.If a difference value is close to 0, movement of the subject acrossframes is small, and it can be estimated that the subject is stationary.Also, if a difference value becomes positive it can be estimated thatthe subject is moving in an approaching direction. If a difference valuebecomes negative it can be estimated that the subject is moving in areceding direction. Further, if a difference value is large it can beestimated that movement speed of the subject is fast while if thedifference values small it can be estimated that the speed of thesubject is slow.

In order to estimate subject movement over a long period, the movementstate estimating section 36 b applies a plurality of subject movementestimation results that have been estimated over the above describedshort period (across two continuous frames) to a plurality of framesgreater in number than the two frames within a fixed period.Specifically, by analyzing continuity of subject movement across aplurality of consecutive two frames that has been estimated, it isdetermined whether the subject is stationary, whether the subject ismoving in a receding or approaching direction, and whether the subjectis moving irregularly etc.

One example of chronological change in difference values for Q1˜Q3 isshown in FIG. 11 . The horizontal axis of the graph shown in FIG. 11corresponds to frame number (elapsed time), with F1 being a time when afirst frame is imaged, for example, and F2 being a time when a secondframe is imaged (the same also applies to F3˜F12). Also, the verticalaxis of the graph represents difference value level for Q1, Q2 and Q3,with a Q1 difference at time F2 being a difference value of Q1 for timesF2 and F1, a Q2 difference at time F2 being a difference value of Q2 fortimes F2 and F1, and Q3 difference at time F2 being a difference valueof Q3 for times F2 and F1. Each point in the graph of FIG. 11 representsrespective difference values for Q1, Q2, and Q3 between two frames,being the current frame and the previous frame. By determining whatchange there is in difference values of Q1˜Q3 across specifiedcontinuous frames, it is possible to estimate long-term subjectmovement.

With the example shown in FIG. 11 , in the earlier period T1(substantially from times F2 to F5), difference values for Q1˜Q3 changepositively and negatively across 0, and it can be determined that thesubject is moving irregularly. Also, in the later period T2(substantially from times F6 to F12), difference values for Q1˜Q3 arecontinuously positive, and it can be generally determined that thesubject is moving in an approaching direction.

FIG. 12 shows a method of estimating subject movement, by comparingdifference values for Q1˜Q3 with threshold values (arrowed ranges) todetermine which of states i˜v the difference values correspond to. InFIG. 12 , state determination is executed in order of highest priority,from i to iv. The Q3 arrow represents a lens pulse value range fordetermining what range Q3 difference values correspond to, and the Q2 orQ1 arrow represents a range for determining what range Q2 differencevalues or Q1 difference values corresponds to. Regarding the Q1 or Q2arrow, if a difference value for Q2, in the determination for Q2 or Q1,corresponds to the range of Q2 or Q1 arrow, it means a range fordetermining if a difference value for Q1 corresponds to the range of Q1or Q2 arrow, and that determination is performed for that range. On theother hand, if a difference value for Q1, in the determination of Q2 orQ1, corresponds to the range of Q2 or Q1 arrow, it means a range fordetermining if difference value for Q2 corresponds to the range of Q1 orQ2 arrow state, and that determination is performed for that range.Also, the horizontal axis represents values resulting from havingconverted lens pulse value to a numerical value having units that are aproduct of aperture value F and permissible circle of confusion δ(corresponding to permissible depth). Although difference values areshown as lens pulse values in FIG. 11 , in FIG. 12 the lens pulse valuesare shown by converting to numerical values having units of F×δ.

In FIG. 11 , for example, at time F2 in the second frame, sincedifference values for Q1˜Q3 are all close to 0 (within the range k3˜k2in FIG. 12 ), it is determined to correspond to “iii, static” in FIG. 12. Also, at time F4 of the fourth frame in FIG. 11 , the differencevalues of Q1 to Q3 are in a + direction, and so it is determined tocorrespond to “ii. approaching direction moving body (low speedmovement)” in FIG. 12 . In this way, it is possible to estimate thesubject movement (movement direction, movement speed) based on timeseries change of respective difference values for Q1, Q2, and Q3 acrossa plurality of frames. The method of FIG. 12 is one example and hasmovement state classified into 5 stages, but there may be more detailedsetting of threshold values and classification into more stages.

FIG. 13 shows an example of making results of having applied quartilesinto a graph in time series order. The vertical axis of the graph islens pulse, and the horizontal axis is time corresponding to framenumber. FIG. 13 shows chronological change in the quartiles Q1˜Q3, andin Min and Max, with in focus range IFR that has been made a diagonalline representing actual in focus range of the subject. With the exampleshown in FIG. 13 it is a case where subject movement advances at aconstant rate from the infinity end to the close-up end direction. Withthis example, values of Q3 and Q2 are located in the actual in focusrange of the subject, and inclination of a graph of Q3 representingdifference value of Q3 substantially matches the inclination of thefocusing range IFR.

If subject movement has been estimated in step S15, next, ranging areasare selected (S17). Here, the ranging area selection section 36 cselects ranging areas. With a method that is generally performed as amethod for selecting ranging areas, for example, an area exhibiting aclosest range lens pulse value may be selected from among lens pulsevalues of ranging areas that have not been excluded and havereliability, and also, ranging areas that have been designatedinstructed by the photographer may be tracked, and ranging areas may beselected based on results of face detection etc. Also, in the case ofperforming moving body prediction, areas exhibiting a lens pulse valuethat is closest to a moving body prediction value may be selected. Here,defocus amounts of ranging areas that have been selected are output tothe focal position prediction section 37. It should be noted thatdefocus amounts that are output to the focal position prediction section37 may be data of a single ranging area that has been selected, and if aplurality of ranging areas have been selected data of these rangingareas may be made into a single set of data.

Once ranging areas have been selected, next, focal position predictionprocessing is performed (S19). Here, the focal position predictionsection 37 predicts focal position of a subject that is a trackingtarget at the time of actual shooting, based on defocus amount of aselection area that has been output from the second AF calculationsection 36, and estimated subject movement. Since history informationfor defocus amount, that there is always a certain number of, is used inmoving body estimation computation, in a case where subject movement haschanged suddenly (for example, change from receding to approaching), itis difficult to deal with immediately. Accuracy of moving bodyestimation computation is therefore improved by using reliabilitydetermination results for the quartiles (refer to S13), anddetermination results for estimated subject movement (refer to S15). Asa method of improving moving body estimation computation accuracy, allor either (one or a plurality) of the following (1) to (4) areimplemented. It should be noted that detailed operation of focalposition prediction processing will be described later using FIG. 5A andFIG. 5B.

(1) Subject movement that has been estimated using quartiles, andinclination of a prediction equation (representing subject movement)that has been calculated using moving body estimation computation, arecompared, and if the advancing directions of the subject are different,for a computation method for the prediction equation, a linear equationis not selected, and an average value for a previous specified number oftimes, or a current value (most recently acquired value), is selected(refer to S55 and S57 in FIG. 5B). In the case of this state, there is apossibility that history information for defocus amount is incorrect,and so a linear equation with which there is a possibility ofincorrectly predicting predicted position is not used, and average valueand current value (most recently acquired value) are used. In this wayit is possible to prevent erroneous moving body prediction AF.

(2) In a case where reliability of quartile application results is high,if it has been determined that there has been change in subject movement(from moving to static, from static to moving, or from approaching toreceding, or from receding to approaching), a number of items of historyinformation to be used when performing calculation with moving bodyestimation computation is reduced (refer to S41 and S43 in FIG. 5A).Moving body estimation computation predicts in-focus position of asubject after a predetermined time (at the time of actual shooting),using previous defocus amount. This means that a greater number of datafor previous defocus amount will generally improve accuracy. However, ina case where there is a high possibility that subject movement ischanging, the number of items of data for defocus amount that are usedin moving body estimation computation is reduced, and by not usinghistory information on the past side, it is possible to calculatepredicted position with good accuracy using only the recent informationcloser to the latest.

(3) When reliability of quartiles is high, if it is determined that asubject that has been estimated using quartiles is moving at high speed,a prediction equation for moving body estimation computation is not alinear equation, and is changed to a quadratic equation (refer to S59and S61 in FIG. 5B). In this state, since subject movement is highspeed, by using a quadratic equation as the prediction equation, and nota linear equation, it is possible to calculate an appropriate predictedposition of the subject significantly advancing to movement direction.

(4) When reliability of application results of quartiles is high, if itis determined that a subject that has been estimated using quartiles ismoving at low speed, a prediction equation for moving body estimationcomputation is not a quadratic equation, and is changed to a linearequation (refer to S59 and S63 in FIG. 5B). In this state, since subjectmovement is low speed, by using a linear equation as the predictionequation, and not a quadratic equation, it is possible to calculate ahighly accurate predicted position.

If moving body prediction has been performed in step S19, the flow forAF operation is terminated. If a state where the photographer haspressed the release button down halfway is maintained, and the imagesensor 21 acquires image data for one frame, the processing of steps S1to S19 are repeated again. Also, if the photographer presses the releasebutton down fully, the lens control section 13 moves the focus lens tothe focal position that was predicted in step S19, and actual shootingis performed.

In this way, in the flow for AF operation, quartiles are obtained (S11),subject movement is estimated using these quartiles (S15), and movingbody prediction is performed using this estimated subject movement(S19). Quartiles can make processing scale small even with a huge numberof ranging areas (lens pulse values), and so it is possible to calculatein a short time. Since it is possible to estimate subject movement withhigh-speed processing and with good accuracy using these quartiles, itbecomes possible to perform moving body prediction AF with goodaccuracy.

Also, with the flow for AF operation, since lens pulse region divisionis performed before obtaining quartiles, and lens pulse values that areconsidered to be for unwanted subjects and ranging errors are excluded(refer to S5 to S9), it is possible to calculate quartiles with goodprecision. Further, since reliability of the quartiles that have beencalculated is evaluated (refer to S13), and quartiles with lowreliability are not used, it is possible to estimate subject movementwith good accuracy.

It should be noted that in the flow for the AF operation, beforecalculating quartiles lens pulse region division is performed (refer toS5), and preprocessing is performed, such as excluding unnecessary lenspulse regions at the close-up end and at the infinity end (refer to S5to S9). This is done in order to calculate quartiles with good accuracy,but if it is possible to ensure a certain degree of accuracy withoutexecuting this preprocessing it may be suitably omitted, or the contentsof this preprocessing may be simplified. Also, before estimating subjectmovement in step S15, reliability of the quartiles has been determined,but if, in step S11, it is possible to calculate quartiles with acertain degree of reliability ensured, determination of the reliabilityof the quartiles (S13) may be omitted.

Also, in the flow for AF operation quartiles have been used asrepresentative values that include statistical dispersion of rangingvalues, but this is not limiting, and values such as average value andstandard deviation value etc. of ranging values (lens pulse values) ofspecified frames may also be used. In a case where statisticaldispersion is made standard deviation or mean deviation, representativevalues that includes statistical dispersion may become averagevalue±standard deviation, or average value±mean deviation. For exampleinstead of the above described quartiles Q1, Q2 and Q3, (averagevalue-standard deviation), average value, and (average value+standarddeviation) may be used, and exactly the same processing may beperformed. Instead of estimating reliability of the quartiles, ahistogram may be created from history information of standard deviation,and if relative frequency of specified classes is greater than athreshold value it may be determined that reliability is high. Also, ina case where standard deviation of a particular frame is smaller than aspecified threshold value, it may be determined that reliability of thatframe is high. As a method of estimating subject movement, instead ofdifference values of quartiles across frames, difference values acrossframes for average value±standard deviation may be used. Also, differentvalues across frames may also be applied to a method that estimatescontinuous subject movement using different values across a plurality offrames.

Next, operation of the statistical processing in steps S5 to S11 will bedescribed using the flowchart shown in FIG. 4 . If the flow forstatistical processing is commenced, first, region division (lens pulseregion division) is performed (S21). Here, the statistical processingsection 36 a performs region division using lens pulse values of eachranging area of a frame that has been acquired this time. Operation ofthis region division is the same as for step S5 in the flow of FIG. 3 ,and so detailed description is omitted.

Once division into lens pulse region has been performed in step S21, itis next determined whether areas that are included in lens pulse regionsat the close-up end are few in number (S23). These steps S23 and S25correspond to processing for close-up end lens pulse region exclusion instep S7 (refer to FIG. 3 ) that was described previously. In this stepS23, the statistical processing section 36 a compares lens pulse valuescontained in close-up end lens pulse regions, among the lens pulseregions that were divided in step S21, with threshold values. If theresult of determination in step S23 is that the lens pulse values thatare included in close-up end lens pulse regions are few in number (lessthan a threshold value), corresponding lens pulse regions are excluded(S25). Here, since there are only a few lens pulse values contained inthe close-up end lens pulse regions, these lens pulse values areexcluded from targets of calculation for quartiles (refer to FIG. 7 ).It should be noted that the lens pulse values that have been excludedare not used in moving body prediction.

If corresponding lens pulse regions have been excluded in step S25, orif the result of determination in step S23 was that lens pulse valuescontained in close-up end lens pulse regions were not few in number,then next, previous history is compared, and it is determined whether ornot lens pulse regions exist at the infinity end (S27). These steps S27and S37 correspond to processing for infinity end lens pulse regionexclusion in step S9 (refer to FIG. 3 ) that was described previously.In this step. S27, the statistical processing section 36 a compares lenspulse values contained in close-up end lens pulse regions, among thelens pulse regions that were divided in step S21, with previous history.Here, as was described using FIG. 8 , it is determined whether or notthere are lens pulse regions more to the infinity end side than theminimum value LstMin of the previous frame (previous lens pulse valuefurthest to the infinity end side). If the result of determination instep S27 is Yes, the corresponding lens pulse region is excluded (S29).

If the corresponding lens pulse regions have been excluded in step S29,or if the result of determination in step S27 was that infinity end lenspulse region do not exist as a result of comparison with previoushistory, then next, it is determined whether or not a maximum value foreach infinity end lens pulse region is within a range of a thresholdvalue from the closest distance (S31). Here, the statistical processingsection 36 a determines whether or not maximum values for each infinityend lens pulse region are located within a range of a certain fixedvalue Tha (threshold value) from a maximum value PreMax (closest range)for the current frame (refer to Vm3, PreMax, and Tha in FIG. 8 ). If theresult of this determination is No, the corresponding lens pulse regionis excluded (S33).

If the corresponding lens pulse regions have been excluded in step S33,or if the result of determination in step S31 is that maximum value foreach lens pulse region at the infinity side is within a threshold valuerange from the closest distance, it is next determined whether or notlens pulse values contained in the infinity end side lens pulse regionare few in number (S35). It was determined in step S23 whether or notlens pulse values contained in close-up end lens pulse regions were fewin number. In this step for the infinity end side lens pulse regions,similarly to step S23, it is determined whether or not lens pulse valuescontained in these lens pulse regions are fewer than a specified number.If the result of this determination is Yes, the corresponding lens pulseregion is excluded (S37).

If corresponding lens pulse regions have been excluded in step S37, orif the result of determination in step S35 was that areas contained inthe infinity end side lens pulse regions were not few in number, next,quartiles are calculated (S39). Here, the statistical processing section36 a performs calculation of quartiles using lens pulse values of eachranging area of a frame that has been acquired this time. Calculationoperation for the quartiles is the same as for step S11 in the flow ofFIG. 3 , and so detailed description is omitted. Once quartiles havebeen calculated, the originating flow is returned to.

In the flow for this type of statistical processing, quartiles arecalculated using lens pulse values that have been respectivelycalculated in a plurality of ranging areas. Also, in this calculation ofquartile, there is division into a plurality of lens pulse regions, andlens pulse regions meeting specified conditions are excluded. Byperforming this processing it is possible to exclude lens pulse valuescorresponding to crossing subjects and unwanted subjects, and lens pulsevalues that contain a large ranging error, and it is possible to improvethe accuracy of quartile calculation. Also, since quartiles can becalculated without using difficult computations, it is possible toobtain the quartiles with short time processing.

It should be noted that in the flow for statistical processing, it hasbeen determined whether or not there are lens pulse regions to beexcluded using three conditions for infinity side lens pulse regions(refer to S27, S31 and S35), but not all of these conditions need to bedetermined, and as long as it is possible to ensure desired accuracysome steps may be omitted, and other conditions may be added. Similarly,for the close-up end lens pulse regions also, conditions may be omittedand other conditions may be added. Also, a method for lens pulse regiondivision is not limited to the method that was described using FIG. 6 ,and another method may be used.

Next, detailed operation of the focal position prediction processing ofstep S19 (refer to FIG. 3 ) will be described using the flowcharts shownin FIG. 5A and FIG. 5B. If the focal position prediction processing iscommenced, it is first determined whether or not there is change inestimated subject movement (S41). Here, the focal position predictionsection 37 performs determination by referencing results of subjectmovement estimation in step S15. For example, as was described usingFIG. 12 , the movement state estimating section 36 b estimates movementstate of the subject based on history of quartile. Therefore, in thisstep change in subject movement is determined based on whether or notthere is change in subject movement in accordance with estimationresults by the movement state estimating section 36 b.

If the result of determination in step S41 is that there is change insubject movement, a number of data points for prediction is reduced(S43). Data for prediction means lens pulse value that have beenacquired previously that are used for moving body prediction. In thecase of performing moving body prediction, position of the focus lens(lens pulse value) where focus is achieved on amain subject at the timeof actual shooting is calculated using a plurality of previous lenspulse values. Generally, it is possible to perform prediction withhigher precision if a greater number of lens pulse values are used inmoving body prediction. However, in a case where there is change insubject movement, lens pulse values from a long time before the currentpoint in time are not conducive to prediction accuracy. Therefore, instep S43, the number of history data for prediction is reduced, so thatlens pulse values used in prediction are only lens pulse values thatwere acquired close to the current time.

If the number of data points for prediction has been reduced in stepS43, or if the result of determination in step S41 was that there was nochange in estimated subject movement, next, the number of data forprediction is determined (S45). Here, the focal position predictionsection 37 determines a number of data for prediction that will be usedin moving body prediction. If the number of data points for predictionwas not changed (S41 No), default values are adopted to the number ofdata for prediction. If the number of data points for prediction waschanged in step S43, the number of data points after change is made thenumber of data for prediction.

Next, exclusion of data for prediction is performed (S47). Here, datathat is not necessary for performing calculation of a predictionequation, namely lens pulse values that are significantly away from theprediction equation and lens pulse values that exceed a specifiedthreshold value, from among defocus amounts (lens pulse values) ofselected areas that were calculated by the first AF calculation section35 and output from the second AF calculation section 36, are excluded.

Next, calculation is performed with a linear prediction equation (S49).Here, the focal position prediction section 37 calculates position ofthe focus lens (lens pulse value) at the time of actual shooting usingcollinear approximation, that is, using a linear prediction equation andprevious defocus amounts (lens pulse values). Also, only a linearprediction equation may be calculated.

Next, calculation is performed with a quadratic prediction equation(S51). Here, the focal position prediction section 37 calculatesposition of the focus lens (lens pulse value) at the time of actualshooting using quadratic curve approximation, that is, using a quadraticprediction equation and previous defocus amounts (lens pulse values).Also, only a quadratic prediction equation may be calculated.

Next, it is determined whether or not reliability of the quartile ishigh (S53). Since reliability of quartiles is determined in step S13,determination here is based on results of that reliabilitydetermination. If the result of this determination is that reliabilityof the quartiles is high, it is next determined whether or not directionof estimated subject movement, and gradient of a prediction equation,are different (S55). Direction of subject movement (approachingdirection or receding direction) is estimated in step S15. Also,gradient (positive/negative) of a prediction equation that has beencalculated in steps S49 and S51 corresponds to subject movementdirection (approaching direction/receding direction) based on theprediction equation.

If the result of determination in step S55 is that the subject movementdirection and prediction equation gradient are different, a predictionequation is not used in calculation of focal position, and insteadaverage value or current position is used as focal position (S57). Ifdirection of subject movement that has been estimated based on quartiledoes not match subject movement direction derived using moving bodyprediction, it is determined that the accuracy of the predictionequation is low. Therefore, instead of using the prediction equation,average values of previous lens pulse values (defocus amounts), orcurrent lens pulse value (lens pulse value that has been calculated mostrecently) are adopted as positions (lens pulse values) for the focuslens at the time of actual exposure. It should be noted that although itis determined whether estimated subject movement direction matchesdirection of movement represented by gradients of the predictionequation, comparison of absolute values, namely magnitude of subjectmovement and magnitude of prediction equation inclination, may also betaken into consideration.

On the other hand, if the result of determination in step S55 is thatthe subject movement direction and the gradient of the predictionequation are not different, and that they substantially match, it isnext determined whether or not estimated subject movement is high-speed(S59). Since it is also determined in step S15 whether or not subjectmovement is high speed (refer to high speed movement of i or v in FIG.12 ), determination is based on the result of this determination.

If the result of determination in step S59 is that the subject is movingat high speed, a quadratic prediction equation is selected (S61). If thesubject is moving at high speed, prediction computation can performprediction with good accuracy using a quadratic equation, and so aquadratic equation is selected, or the computation results of step S51are adopted.

On the other hand, if the result of determination in step S59 is thatthe subject is not moving at high speed, a linear prediction equation isselected (S63). Since the subject is not moving at high speed,prediction computation can perform prediction with good accuracy using alinear equation, and so a linear prediction equation is selected, or thecomputation results of step S49 are adopted.

If a prediction equation has been selected in step S61 or S63, or if itwas not a prediction equation in step S57 and instead average value orcurrent position were made target focus lens position, or if the resultof determination in step S53 was that reliability of quartiles was nothigh, focal position is determined (S65). A focus lens position that wasdetermined in step S57 is determined to be focal position.Alternatively, focus lens position where focus will be achieved at thetime of actual shooting is calculated based on a prediction equationthat was selected in steps S61 or S63, and data for prediction, and thefocus lens position that has been calculated is determined to be a focalposition. Also, if, in the determination of step S53 the reliability ofthe quartiles was not high, focus lens position is calculated using amoving body estimation computation method that would have been performedconventionally (ranging area representing the most recent defocus amount(lens pulse value) selected in a moving body prediction equation). Oncefocal position has been determined, the originating flow is returned to.

In this way, in the flow for focal position prediction processing, sincemoving body prediction is performed using subject movement that wasestimated in step S15 (refer, for example, to S41, S55, and S59), it ispossible to improve the accuracy of moving body estimation computation.Subject movement can be estimated based on quartiles calculated based onranging values (lens pulse value) that have been calculated in aplurality of ranging areas.

It should be noted that in order to predict focal position, in the flowof FIG. 5A and FIG. 5B, various determinations have been performed(refer, for example, to S41, S53, S55, S59 etc.), but some of thesedeterminations may be omitted, and other determination conditions may beadded.

Next, applying of the present invention to another device will bedescribed using FIG. 14 . In FIG. 1 to FIG. 13 description was given fora case where the present invention was applied to a camera. However, thepresent invention is not limited to a camera and it can be applied toany device that focuses on an object that is moving. FIG. 14 shows anexample of applying the present invention to an observation device thatperforms observation of cells that move in a floating manner, in acultivation device that performs suspension cultivation of cells. Insuch a cultivation device, at the time of observing cell cultivation itis difficult to focus on cells that are moving while being suspended(for example, moving to approach in a linear direction, or in a recedingdirection), images are blurred, and it is difficult to appropriatelyperform observation.

FIG. 14 shows one example of a cell culture device 41 to which thepresent invention has been applied. Refer to Japanese patent laid-openNo. 2020-141589 (US Patent Application Publication No. 2020/0283721)regarding the detailed structure of this cell culture device. US PatentApplication Publication No. US 2020/0283721 is incorporated herein byreference. The cell culture device 41 comprises a supply section 46 thatsupplies a culture medium W to a culture vessel 43 that is capable ofholding the culture medium W and cells S, a discharge section 47 thatdischarges culture medium W and cells S from the culture vessel 43, anobservation section 49 that measures cell density within the culturemedium W inside the culture vessel 43, an agitation mechanism 51 foragitating the culture medium W inside the culture vessel 43, and acontrol section (not shown) for controlling the supply section 46,discharge section 47, and observation section 49.

As the observation section 49, it is possible to adopt known technologyfor measuring cell density within a culture medium, for example, it ispossible to adopt a structure that provides a retro reflective memberhaving an array of a plurality of minute reflective elements arranged.This observation section 49 comprises, for example, an object lens thatis arranged at the culture vessel 43 side, an imaging section thatphotographs illumination light that has been condensed by the objectlens after having passed through the culture medium W inside the culturevessel 43, and an image analysis section for calculating cell densitywithin the culture medium W inside the culture vessel 43 based on imagesthat have been acquired by the imaging section.

Cells S within the culture medium W are photographed by the imagingsection of this observation section 49. Cell density within the culturemedium W inside the culture vessel 43 is then calculated by the imageanalysis section based on a number of cells S that are contained in animage that has been acquired by the imaging section. In this type ofcell culture device there is a demand to not only obtain density ofcells, but also acquire and store detailed images of specified cells,and create a report as observation data.

In order to observe cells with good accuracy in this cell culture device41, it is necessary to focus on cells with high precision using theobject lens, but if the cells S are moving this is not easy. Also, evenif movement speed of the cells is slow, if there are objects at acomparatively close distance, the image plane movement velocity of thoseobjects can become equal to that of a high speed object at a distance.Therefore, by executing the AF operation that was described in FIG. 3 toFIG. 13 it is possible to perform moving body prediction with cells Sthat are moving as a main subject, and to perform focus with goodaccuracy.

In this way, the focus adjustment device each of the embodiments of thepresent invention calculates statistical values representing dispersionin ranging values of a plurality of areas, and obtains a representativevalue for ranging values included in statistical values representingthis dispersion. Subject movement is then estimated based on time serieschange in this representative value, and reflected in moving bodyprediction. Also, quartiles, for example, are used as statistical valuesrepresenting dispersion, and representative values (refer, for example,to S11 in FIG. 3 ). These quartiles are calculated based on rangingvalues of a plurality of areas for every frame. Further, differencesbetween respective quartiles Q1 to Q3 are calculated across frames.Subject movement is estimated based on change in quartiles Q1 to Q3across a plurality of frames. This estimated subject movement isreflected in moving body estimation computation (order of the movingbody prediction equation, history information etc.).

As has been described above, each embodiment of the present invention isprovided with a plurality of detection areas in an imaging region formedby an optical system, ranging values of an object in the detection areasare detected, and focus adjustment is performed for the object. Whenperforming this focus adjustment, statistical processing is performed onranging values of a plurality of detection areas, representative valuescontaining statistical dispersion are calculated based on thisstatistical processing (refer, for example, to S11 in FIG. 3 ), movementstate of an object is estimated based on time series change of theserepresentative values (refer, for example, to S15 in FIG. 3 ), focalposition relating to the object is predicted based on this estimatedmovement state (refer, for example, to S19 in FIG. 3 ), and focusadjustment is performed based on this focal position that has beenpredicted. With this embodiment, statistical quartiles are applied,directed at ranging values of a plurality of ranging areas, and movementstate of an object is estimated based on time series change in thequartiles, which means that it is possible to estimate movement state ofthe subject without being affected by dispersion such as of rangingerrors etc. In this way, since it is possible to change parameters formoving body prediction based on movement state that has been estimatedwithout being affected by disturbance, it is possible to improve theaccuracy of moving body estimation computation.

It should be noted that in the embodiments of the present invention,processing has been performed mainly using a target lens pulse position(lens pulse value). However, since it is possible to mutuallyinterchange target lens pulse position (lens pulse value) and defocusamount, processing may also be appropriately performed using defocusamount.

Also, with the one embodiment of the present invention, the facedetection section 22 a and tracking section 22 b within the imageprocessing section 22 respectively have a face detection circuit and atracking circuit, but instead of hardware circuits they may also beconfigured as software using a CPU and programs, may be implemented byhardware circuits such as gate circuits that are generated based on aprogramming language described using Verilog, or may be configured usinga DSP (Digital Signal Processor). These sections and functions may alsobe respective circuit sections of a processor constructed usingintegrated circuits such as an FPGA (Field Programmable Gate Array).Suitable combinations of these approaches may also be used. The use of aCPU is also not limiting as long as a processor fulfills a function as acontroller.

Also, regarding each of the sections with the AF calculation section 23,besides being constructed in the form of software using a CPU andprograms, some or all of these sections may be constructed with hardwarecircuits, or may have a hardware structure such as gate circuitrygenerated based on a programming language described using Verilog, ormay use a hardware structure that uses software, such as a DSP (digitalsignal processor). These sections and functions may also be respectivecircuit sections of a processor constructed using integrated circuitssuch as an FPGA (Field Programmable Gate Array). Suitable combinationsof these approaches may also be used. In cases where calculation ofdefocus amount, reliability evaluation, calculation of contrastevaluation values, and generation of phase difference pixels etc. areoften performed by repeating uniform computational processing, theseoperations may also be configured using hardware circuits. Also, a CPUhas been used as a controller, but the present invention is not limitedto a CPU as long as elements fulfill a function as a controller.

Also, with the one embodiment of the present invention, a device fortaking pictures has been described using a digital camera, but as acamera it is also possible to use a digital single lens reflex camera, amirrorless camera, or a compact digital camera, or a camera for movieuse such as a video camera, and further to have a camera that isincorporated into a mobile phone, a smartphone a mobile informationterminal, personal computer (PC), tablet type computer, game consoleetc., or a camera for a scientific instrument such as a medical camera(for example, a medical endoscope), or a microscope, an industrialendoscope, a camera for mounting on a vehicle, a surveillance cameraetc. In any event, it is possible to apply the present invention to anydevice that repeatedly generates ranging values of an object in aplurality of detection areas, and performs focus adjustment for anobject based on ranging values.

Also, among the technology that has been described in thisspecification, with respect to control that has been described mainlyusing flowcharts, there are many instances where setting is possibleusing programs, and such programs may be held in a storage medium orstorage section. The manner of storing the programs in the storagemedium or storage section may be to store at the time of manufacture, orby using a distributed storage medium, or they be downloaded via theInternet.

Also, with the one embodiment of the present invention, operation ofthis embodiment was described using flowcharts, but procedures and ordermay be changed, some steps may be omitted, steps may be added, andfurther the specific processing content within each step may be altered.It is also possible to suitably combine structural elements fromdifferent embodiments.

Also, regarding the operation flow in the patent claims, thespecification and the drawings, for the sake of convenience descriptionhas been given using words representing sequence, such as “first” and“next”, but at places where it is not particularly described, this doesnot mean that implementation must be in this order.

As understood by those having ordinary skill in the art, as used in thisapplication, ‘section,’ ‘unit,’ ‘component,’ ‘element,’ ‘module,’‘device,’ ‘member,’ ‘mechanism,’ ‘apparatus,’ ‘machine,’ or ‘system’ maybe implemented as circuitry, such as integrated circuits, applicationspecific circuits (“ASICs”), field programmable logic arrays (“FPLAs”),etc., and/or software implemented on a processor, such as amicroprocessor.

The present invention is not limited to these embodiments, andstructural elements may be modified in actual implementation within thescope of the gist of the embodiments. It is also possible form variousinventions by suitably combining the plurality structural elementsdisclosed in the above described embodiments. For example, it ispossible to omit some of the structural elements shown in theembodiments. It is also possible to suitably combine structural elementsfrom different embodiments.

What is claimed is:
 1. A focus adjustment device, that provides aplurality of detection areas in an imaging region formed by an opticalsystem, repeatedly generates ranging values for a physical object in thedetection areas, and performs focus adjustment for the physical objectbased on the ranging values, comprising a processor that has astatistical processing section, a movement state estimating section, aprediction section, and a control section, wherein the statisticalprocessing section subjects ranging values of the plurality of detectionareas to statistical processing; the movement state estimating sectioncalculates representative values that contain statistical dispersionbased on the statistical processing, and estimates movement state of thephysical object based on time-series change in the representativevalues; the prediction section predicts focal position regarding thephysical object based on the movement state that has been estimated; andthe control section performs focus adjustment based on the focalposition that has been predicted, wherein the statistical processingsection applies quartiles as the statistical processing, and wherein themovement state estimating section includes an interquartile range as therepresentative values, determines reliability based on time serieschange of an interquartile range, and if it is determined that there isreliability estimates and outputs movement state of the physical object.2. The focus adjustment device of claim 1, wherein: the movement stateestimating section estimates movement direction of the physical objectbased on time-series change in the representative values.
 3. The focusadjustment device of claim 1, wherein: the movement state estimatingsection estimates movement speed of the physical object based ontime-series change in the representative values.
 4. The focus adjustmentdevice of claim 1, wherein: the statistical processing sectioncalculates deviation and/or average values.
 5. The focus adjustmentdevice of claim 4, wherein: the representative values contain theaverage value and the deviation.
 6. The focus adjustment device of claim4, wherein: the movement state estimating section determines reliabilitybased on time series change of the representative values, and if it isdetermined that there is reliability estimates and outputs movementstate of the physical object.
 7. The focus adjustment device of claim 1,wherein: the movement state estimating section makes the representativevalues a first quartile, or a second quartile, or a third quartile. 8.The focus adjustment device of claim 1, wherein: the movement stateestimating section determines reliability based on degree of dispersionof the interquartile range, and/or size of the interquartile range, andor number of the ranging values that are valid contained in theinterquartile range.
 9. A focus adjustment method, that provides aplurality of detection areas in an imaging region formed by an opticalsystem, detects ranging values for a physical object in the detectionareas, and performs focus adjustment for the physical object,comprising: subjecting ranging values of the plurality of detectionareas to statistical processing; calculating representative values thatcontain statistical dispersion based on the statistical processing, andestimating movement state of the physical object based on time-serieschange in the representative values; predicting focal position regardingthe physical object based on the movement state that has been estimated;performing focus adjustment based on the focal position that has beenpredicted; in the statistical processing, applying quartiles; and inestimation of the movement state, determining reliability based on timeseries change of an interquartile range, and if it is determined thatthere is reliability, estimating and outputting movement state of thephysical object.
 10. The focus adjustment method of claim 9, furthercomprising: in estimating the movement state, estimating movementdirection of the physical object based on time-series change in therepresentative values.
 11. The focus adjustment method of claim 9,further comprising: in estimating the movement state, estimatingmovement speed of the physical object based on time-series change in therepresentative values.
 12. The focus adjustment method of claim 9,further comprising: in the statistical processing, calculating deviationand/or average values.
 13. The focus adjustment method of claim 12,wherein: the representative values contain the average value and thedeviation.
 14. The focus adjustment method of claim 12, wherein: inestimation of the movement state, determining reliability based on timeseries change of the representative values, and if it is determined thatthere is reliability estimating and outputting movement state of thephysical object.
 15. The focus adjustment method of claim 9, furthercomprising: in the movement state estimation, making the representativevalues a first quartile, or a second quartile, or a third quartile. 16.A non-transitory computer-readable medium storing a processor executablecode, which when executed by at least one processor, the processor beingarranged in a focus adjustment device that is provided with a pluralityof detection areas using an imaging region formed by an optical system,and that detects ranging value of a physical object in the detectionareas, performs a focus adjustment, the focus adjustment methodcomprising: subjecting ranging values of the plurality of detectionareas to statistical processing; calculating representative values thatcontain statistical dispersion based on the statistical processing, andestimating movement state of the physical object based on time-serieschange in the representative values; predicting focal position regardingthe physical object based on the movement state that has been estimated;performing focus adjustment based on the focal position that has beenpredicted; in the statistical processing, applying quartiles; and inestimation of the movement state, determining reliability based on timeseries change of an interquartile range, and if it is determined thatthere is reliability, estimating and outputting movement state of thephysical object.
 17. The focus adjustment method of claim 9, furthercomprising: determining reliability based on degree of dispersion of theinterquartile range, and/or size of the interquartile range, and ornumber of the ranging values that are valid contained in theinterquartile range.